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Introduction 


The process of accrediting institutions of higher education in the 
U.S. embraces two principles: the system provides for self-regulation con- 
ducted by peers; and the starting point of the procedure is the institution’s 
academic mission and goals. In practice, the principle of peer assessment 
means that an institution “must demonstrate that it meets the collective 
standards of its peers” (Braskamp, Poston, Wergin, 1). With the frame- 
work of the process being the institution itself, the operative word is 
“standards” rather than standardization. The principle of institutional 
autonomy thus protects the individuality of the institution and allows for 
a wide variety of colleges and universities. 2 The evaluation of a study 
abroad program might be designed according to these same principles, 
but international and cross-cultural program components introduce three 
specific challenges. First, we must articulate the program’s academic mis- 
sion and goals in terms of cross-cultural learning experiences, among 
them, language facility, field study and guided field trips, and access to 
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local educational, cultural and political institutions. Second, we must con- 
sider non-academic program elements such as predeparture information 
and on-site orientation, housing, provisions for health care, and opportu- 
nities for interactions between American students and members of the 
host country. Third, we must negotiate the differences in academic cul- 
tures, demonstrated by pedagogical practice, student assessment, and 
grading. 

A uniform set of educational standards and a protocol for program 
evaluation do not exist for most study abroad programs. The Middle 
States Association of Colleges and Universities is the only one of the six 
regional accrediting associations in the U.S. that evaluates the study 
abroad programs of its members as part of the accreditation process 
(CHE/MSA, 1). NAFSA: Association of International Educators, offers 
guidelines for evaluation, but is not vested with the authority to enforce 
them. 3 Yet, the reasons that support accreditation in higher education 
likewise apply to the need for rigorous and systematic evaluation of study 
abroad programs. Among them are: assisting the prospective student in 
identifying an acceptable program; assisting institutions in determining 
the acceptability of transfer credits; creating goals for the improvement of 
programs; and involving faculty and staff in institutional evaluation and 
planning. (Sims, 42). Study abroad providers, while not accountable to the 
general public as are public institutions of higher education, should be 
held accountable to the students they enroll. Dr. Minna Weinstein, Senior 
Associate Executive Director, Middle States Association, explained, “Our 
members expend a lot of time and energy to evaluate programs for small 
groups of students, but it’s our responsibility to make sure programs are 
good even if they only serve 30 students.” Principles and policies of eval- 
uation would contribute to the professionalism of study abroad, assuring 
administrators, faculty members, and students of its academic value. The 
net result would support the growth of study abroad, the “rapid develop- 
ment of the capacity to address the international dimensions of U.S. high- 
er education” (Gagliano, 326). 

This paper describes how the authors worked with a Task Force to 
develop the IES Model Assessment Practice (MAP), a set of detailed cri- 
teria to use in evaluating IES study abroad programs, and how IES is 
beginning to implement MAP. Two sets of theories provided the frame- 
work for the project: current thinking about educational program evalua- 
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tion and assessment in the U.S., including current definitions of academ- 
ic quality; and organizational learning, whereby the processes of planning 
and decision-making involve members of the organization in gathering, 
sharing, and interpreting information. The 18-month project was spon- 
sored by IES, The Institute for the International Education of Students 
(formerly the Institute of European Studies). With this publication, we 
propose that study abroad practitioners join a dialogue about program 
evaluation and share models of good practice for the benefit of our stu- 
dents. 

IES Programs 

IES, founded in 1950, is a consortium of American colleges and uni- 
versities with 50 affiliates and 75 associate members, and runs semester, 
full-year, and summer programs for primarily American undergraduates 
in 17 locations in Europe, Asia, Australia, and Latin America. IES acade- 
mic programs cover a range of disciplines, including anthropology, area 
studies, business, economics, environmental studies, history, literature, 
intensive language, music theory, political science, psychology, sociology, 
and women’s studies. All programs offer internships in at least one field — 
for example, business or education — and several programs offer a variety 
of internships in the arts, politics, NGOs, and the media. A supporting 
seminar is required of students with an internship placement. Most pro- 
grams are designed as full immersion, requiring two years of language 
instruction at the college level for admission and offering their curricula 
in the language of the host country. Other programs admit students at 
beginning and low- intermediate levels of the target language; in these 
cases, intensive language is required on site, and English is the language 
of instruction for area studies courses. Academic partnerships enable stu- 
dents to register at one or more local universities and, in some cases, at 
music conservatories and schools specializing in studio art. 

Staff members and instructors are drawn from the host country; in 
two cases, directors are American citizens with longstanding academic 
affiliations in the host country. Many directors are practicing academics 
and teach an IES course as well as supervise the administration of their 
programs. All courses are taught by nationals of the host country, many of 
whom are full faculty members at partner institutions. Academics/profes- 
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sionals teach courses in the arts and business. 

IES programs also offer student support services for both academic 
and non-academic activities. Examples of the former include academic 
counseling regarding registration in IES courses and courses at the local 
university; privileges at university libraries and other local libraries; and 
computer facilities. Non-academic activities that are organized and super- 
vised by IES staff members include housing placement, guided field trips, 
orientation, and events organized by the Center to facilitate engagement 
in the local culture. 

IES Model Assessment Practice(MAP): 
Development 

A Task Force was formed to develop MAP that consisted of two rep- 
resentatives of the IES Board of Trustees, three faculty members from IES 
affiliate members, and staff members of the IES Academic Department in 
Chicago, one of whom is one of the authors. The Task Force was led by a 
consultant in the field of higher education evaluation, also one of the 
authors. In its initial meeting, the Task Force agreed on the principles, 
methodology and scope of the project. 

The project was conceived for the purpose of developing a set of inclu- 
sive and specific criteria to measure program quality, to be used by IES 
directors and staff members for improving existing programs. In addition to 
setting this operational strategy in motion, the project’s secondary goal was 
to establish a conceptual framework of quality that would serve IES in its 
review of existing programs and the development of new programs. The 
premise of MAP was to preserve the differences among program sites, or 
Centers, based on each program’s purpose and curriculum, the culture of the 
host country, and the type of student for whom the program was designed. 
This premise was explained by the following principles: 

• Integrating the Center academic programs with the local culture is 
a more effective and powerful learning environment for IES students 
than alignment with an American set of expectations of the learning 
environment. 

• Diversity of academic programs across IES, rather than uniformity 
and conformity, is critical to ensure that a Center represents the local 
culture. 


104 



Joan Gillespie, et. al. 


• High standards of quality across IES rather than standardization are 
the best way to achieve academic excellence at each Center. (IES, 
“Assessing,” 2) 

Information was collected from the major stakeholders in IES pro- 
grams — IES directors, on-site staff members, instructors, current stu- 
dents, and campus coordinators at IES affiliate and associate member 
schools. In addition, data such as program and course evaluations by stu- 
dents and reports of cyclical program reviews from the IES ongoing inter- 
nal quality assurance system were analyzed. The most important phase in 
gathering information were five site visits, each undertaken by three to 
four members of the Task Force, to IES Centers in Paris, La Plata, 
Freiburg, Vienna, and Beijing, which represent the variety of IES summer, 
semester, and full-year programs on three continents. The reason for these 
extended site visits was to draw on the considerable expertise in study 
abroad of Center directors, staff members, and instructors who are citizens 
of the host country, and to use the programs as laboratories for collecting 
data and testing an evaluation method for study abroad. 

The strategy of data collection and analysis draws from current the- 
ory in program evaluation that establishes a relationship between system- 
atic inquiry and organizational learning. The phase of inquiry requires 
that employees figure prominently in the process of asking questions and 
gathering information, not only to benefit the organization through their 
expert knowledge, skills, and ideas but also to give employees a sense of 
ownership in the organization. This ownership ultimately will affect 
employees’ commitment to implementing findings and recommendations 
(Preskill and Torres, 12). The concept emphasizes the process of both indi- 
vidual and organizational learning; employees develop the habit of 
exchanging ideas and information and the organization puts systems into 
place that make use of this exchange. The planning of the IES MAP rep- 
resented the beginning of a continuous process of organizational learning 
with regard to the quality of academic programming. The ongoing need 
to collect and share information, then reflect and act on this information, 
was written into MAP in Part IV, Interactions between an IES Center and the 
IES Chicago office , 4 

Task Force members determined to focus on four areas as the means 
for measuring academic quality: the student learning environment; stu- 
dent learning, assessment and the development of intercultural compe- 
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tence; resources for academic and student support; and the interactions 
among staff and faculty at IES Chicago, IES Centers, and colleges and uni- 
versities in the U.S. The first three categories parallel the definition of aca- 
demic quality in American higher education used by the authors and oth- 
ers. 5 These terms are defined as follows: 

I. Student Learning Environment. An educational institution must pro- 
vide engaging and challenging country-specific learning environments for 
students. The quality of the experiences available to students is dependent 
on the structure and effectiveness of courses, out-of-classroom experiences 
including field study and field trips that are organized as part of the pro- 
gram, engagement in cultural activities, and relationships with the facul- 
ty, staff members, and other representatives of the host culture, such as 
host families. In a study abroad setting, the location itself is one of the 
most valuable resources, and the program’s ties with partner universities, 
academic, business, and art groups, host families, and internship sites that 
support a student’s cultural interactions likewise are part of the student 
learning environment. 

II. Student Learning: Assessment and the Development of Intercultural 
Competence. This category, developed over the last decade as an essential cri- 
terion of academic quality in higher education in the U.S., recognizes the 
dual goals of colleges and universities to enhance students’ intellectual 
growth and foster and develop their social and cultural competencies that 
prepare them to live and work in a global society. 6 These goals both define 
an institution’s mission by helping it to establish priorities among desired 
student outcomes and are the means by which the mission is fulfilled 
(Braskamp, 419). Assessment also measures the “value-added,” the effects 
of college upon students’ cognitive and affective development (Nordvall, 
485). “Talent development” is used interchangeably with “value-added” 
and helps define the relationship between mission and assessment — that 
is, if an institution’s mission is to develop the talents of students and fac- 
ulty, then forms of assessment should be integrated into the teaching and 
learning process and be guided by the kinds of talent that the institution 
fosters (Astin, 158; 171). In the development of intercultural competence, 
this principle opens the way for the use of qualitative measures such as 
interviews in the target language or an artistic project to assess a student’s 
level of interaction with the host culture and understanding and accep- 
tance of cultural difference. 
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III. Resources for Academic and Student Support. An educational organi- 
zation must have an infrastructure and financial and human capital to pro- 
vide a high quality learning environment. Criteria are determined in part 
through quantitative measurements: the ratio of on-site staff to students, 
library collections, computer facilities, admission standards, the grade 
point average of enrolled students, and tutorial assistance. Qualitative 
measures include the credentials of faculty, staff, advisors and health and 
safety guidelines. 

IV. Interactions among Staff and Faculty at IES Chicago , IES Centers, and 
colleges and universities in the U.S. While criteria are specific to the admin- 
istrative framework of IES, it might be enlarged to cover an institution’s 
level of communications with a variety of publics, such as government and 
educational agencies at home and abroad. 

The Task Force identified several issues to investigate during the site 
visits in order to determine the criteria of quality assurance according to 
the four categories. The question of balance was critical in defining three 
issues in particular: academic cultures, student activities, and program 
resources. Through observation of classes and interviews with faculty 
members, the Task Force considered the differences in academic cultures, 
that is, the need to introduce American students to the teaching styles and 
assessment practice of the host country while acknowledging American 
students’ learning styles, particularly expectations of specific class assign- 
ments and a high level of interaction with instructors. Interview questions 
with instructors included the following: 

What are your impressions of American students' learning habits? 

Flow would you characterize the level of students' academic and cul- 
tural preparation for a semester in the host country? 

Did you make changes in your syllabus based on the students' prepa- 
ration and comprehension? 

What are the field study assignments and how do they immerse stu- 
dents in the host population and its culture? 

Do you hold office hours or provide conference time for students? 
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Another topic concerned the range of opportunities for cultural inte- 
gration that are provided to students that, at the same time, encourage 
their independence and resourcefulness. A third topic focused on the 
extent to which facilities and resources (particularly computer and educa- 
tional technology) should approximate available technology on U.S. cam- 
puses. These issues were examined through interviews with staff, informal 
meetings with students, a tour of facilities, including facilities at partner 
institutions, housing, and a cultural activity with students or an activity 
or event similar to one that students experience during the semester. 

One other source of information for compiling the criteria of Model 
Assessment Practice was a survey of study abroad coordinators at IES affil- 
iate and associate schools on their perspective of academic quality. This 
survey supplied data on reputation, one of the traditional indicators of 
quality. The reliability of the data depends on those making the judg- 
ments, the criteria they use and their access to evidence that supports the 
criteria (Nordvall, 484). By querying those with the greatest knowledge 
of study abroad programs, the Task Force received data with a high degree 
of reliability. 

MAP in its final form employs the four categories identified at the 
beginning of the project (see Appendix). Criteria in each sub-category are 
descriptive rather than prescriptive, in keeping with the principles that 
guided the project to protect the curricular and cultural diversity of the 
Centers and to promote standards, not standardization. For example, cri- 
teria listed under “Instructional Quality” state: “Class experiences make 
effective use of location,” and: “Language instruction makes effective use 
of field study and local cultural institutions.” The term “effective use” 
allows for latitude in a Center’s decisions regarding the structure and fre- 
quency of these experiences, depending upon the overall curriculum and 
the content of individual courses. Students evaluate the “effectiveness” of 
these experiences in program and course evaluations. 

MAP consists of four other evaluative procedures, three consisting of 
formative data and one of summative data. 7 The criteria for quality assur- 
ance served as the conceptual framework for course evaluations and gen- 
eral evaluations of a program, both completed by students. These evalua- 
tive instruments already were in use and were revised from two perspec- 
tives: to reflect the assessment criteria, thereby giving students the oppor- 
tunity to indicate how well they are being met, and to offer instructors 
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and program staff the means to assess students, albeit anonymously. 
Course evaluations, focusing on the instructional process and the cultural 
context, question students about their understanding of course content, 
the academic rigor, and the use of the local environment. While tradi- 
tional assessment tools such as course examinations and research assign- 
ments also measure a student’s understanding of content, they do not 
reveal a student’s expectations of academic rigor or his or her insights into 
the local culture. By making these assessments, course evaluations are 
intended to be a planning aid to instructors to balance course content with 
student expectations of knowledge and comprehension, which in turn 
facilitate cultural analysis. This use focuses academic quality on academic 
processes in the classroom and places part of the responsibility for improv- 
ing academic quality in the hands of instructors (Nordvall, 486). 

The general program evaluation serves as a form of student self- 
assessment, particularly in the categories of interpersonal and intraper- 
sonal skills. Questions address a student’s independence, level of adapta- 
tion, understanding of his/her own cultural values, self-knowledge, prob- 
lem-solving skills, and tolerance of diversity. These affective and behav- 
ioral areas of growth are central to the experience of study abroad, yet are 
difficult to quantify. These evaluations are analyzed in tandem with eval- 
uations conducted by home institutions when their students return to 
campus. 

Finally, the assessment criteria serve as the basis for the in-depth 
five-year reviews of each program, conducted by a committee composed of 
faculty representatives of affiliate members and of instructors and students 
at the Center. These program reviews have been in place at IES since 
1974, and will continue as an essential step in monitoring and improving 
academic quality. 

A set of summative data was developed as part of the ongoing sys- 
tem of program evaluation by program directors and the Chicago office. 
Statistical profiles of each Center summarize the most significant infor- 
mation on student admissions, revenues and expenses, the academic pro- 
gram, human resources, and physical resources. Purely quantitative, the 
profiles offer the basis for making administrative decisions about admis- 
sions, recruiting, programming, staffing, space needs, and funding. The 
profiles are updated each semester and their use for planning purposes 
applies in most cases to the coming academic year. 
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Implementation of MAP 

After the criteria of quality assurance were drafted, the process of 
collecting data and analyzing information continued, in keeping with the 
concept of organizational learning. The next step was a self-study accord- 
ing to the criteria that was conducted by program directors and their staff. 
Because MAP allows for differences among cultures and programs, the cri- 
teria do not constitute a grading system or even a checklist. Instead, the 
self-study requires that program directors and faculty focus on those cri- 
teria that apply to their program, report on how they meet the criteria, 
and, if they do not meet them, how they would propose to do so. 

The first section, Student Learning Environment, contains two 
major divisions, one related to academic programming and another focus- 
ing on out-of-classroom activities that are devoted to a student’s cultural 
integration. The criteria devoted to Instructional Quality generated 
responses in the self-study that describe the difficulty of negotiating 
between American students and faculty of the host country, each group 
with its expectations of student preparation and teaching methods. IES 
Dijon explained: “Faculty have been hesitant about setting their level of 
expectation. They find that students have the potential but are easily dis- 
tracted by the environment” (IES, “Self-Assessment,” 4). IES Freiburg 
responded: “Faculty have to consider very diverse language levels of stu- 
dents Faculty-staff meetings are held once per semester, where teach- 

ing methods and the question of standardization (how Americanized 
can/must a course be without jeopardizing educational goals like inter- 
cultural academic competency) are discussed” (IES, “Self-Assessment,” 5). 
The challenge of negotiating the differences between students and facul- 
ty led to the decision at IES Chicago to discuss this issue in a meeting of 
all directors and reach agreement about balancing the needs and expecta- 
tions of both parties. 

Self-assessments in another category in Student Learning 
Environment, Curricular Design and Integration, revealed the extent to 
which courses take advantage of the students’ presence in the country to 
enhance their knowledge of political, cultural, and social life. Freiburg 
and Vienna specifically identified enrollment in partner institutions as “an 
absolute guarantee that students become integrated in community life as 
far as their individual social skills permit” (IES, “Self-Assessment,” 8), a 
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statement that confirms the priority given to academic advising for uni- 
versity enrollment in full-immersion programs. This category also under- 
scored the differences between programs and the relevance of standards, 
not standardization. In the self-assessments according to the criterion, 
“Course content and curriculum adequately reflects the variety of cultures 
of the host country,” IES La Plata responded: “Argentinian common sense 
and academic sense do not consider this as the definition of Argentinian 
culture” (IES, “Self-Assessment,” 9). Similarly, IES Vienna wrote: “A vari- 
ety of cultures in the American sense does not really exist in Austria,” and 
IES London explained: “It is more important that students learn the dom- 
inant culture first, then diverse cultures. In England, the challenge is to 
look outside London; London itself is diverse but not representative of the 
rest of Britain” (IES, “Self-Assessment,” 9). Thus, while the criterion 
might not apply to all countries, its inclusion is significant, particularly 
for countries such as France and Germany that are addressing the politi- 
cal, social, and economic challenges of immigrant communities. 

One theme in the four major categories of MAP focuses on the 
development of skills related to intercultural competence, both through 
coursework and through out-of-classroom experiences (Moore and Ortiz, 
47-50). This focus is consistent with the IES mission and its current 
efforts to develop program support to foster these skills through cogni- 
tive, interpersonal, and intrapersonal growth. The targeted outcome is 
that students develop the ability to understand and critique their own cul- 
ture. Thus, one of the criteria is: “Approaches to course material should 
incorporate an analysis of cultural differences.” Several of the self-assess- 
ments were positive, even if such analysis does not constitute a formal 
aspect of the curriculum. For example, IES Tokyo responded: “The incor- 
poration of cultural differences into class is a very natural extension of the 
course material and is a frequent topic of student discussion” (IES, “Self- 
Assessment,” 11). Another criterion in the category of internships stipu- 
lates: “Internships help develop intercultural cognitive and interpersonal 
skills.” The response of IES London drew on student evaluations as the 
basis of its self-assessment that “this development occurs. Internships offer 
a concentrated view of British behaviour, and students must adapt a dif- 
ferent perspective and analytical framework” (IES, “Self-Assessment,” 14). 

Two other programming elements that address cross-cultural learn- 
ing and skills in MAP are on-site orientation and re-entry. The first crite- 
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rion of orientation stipulates that the Center provide information on gen- 
der and race relations in the host country. While several programs 
responded affirmatively, IES Nantes explained, “Students are informed 
about gender relations, but race has not yet been discussed with the entire 
student body. Race relations are dealt with during relevant IES classes or 
on a case by case basis” (IES, “Self-Assessment,” 2). An example of this 
alternative approach in Nantes is an assignment in language classes that 
requires students to discuss race relations in France with their homestay 
families, which meets the intent of the criterion. A re-entry program was 
identified as a priority by some Centers that do not currently offer a for- 
mal session on the challenges and opportunities presented by reverse cul- 
ture shock. Further, mutual support in developing programs on orienta- 
tion and re-entry are included in Part IV, Interactions among IES Chicago, 
IES Centers, and colleges and universities in the U.S. 

Two sections of Part III, Resources for Academic and Student Support 
reflect the current climate of U.S. higher education and study abroad. One 
of these covers the topic of health and safety. Student insurance coverage, 
student access to health care, instructions to students for handling a med- 
ical emergency, and enforcement of the IES policy on sexual harassment 
are among the criteria that Centers must meet. The second sub-category 
is Instructional Technology, available both to students and instructors. 
Guidelines provide for student access to computers, e-mail, and digital 
resources such as the Internet, and technology for faculty use. This latter 
set of criteria represents a significant funding item at Centers where stu- 
dents do not have access to computer facilities at partner universities. 
Options such as shared facilities are being investigated at such programs. 

One goal of the self-study is that program staff members continual- 
ly address the criteria that are relevant to their purpose, curriculum, and 
students. A time limit for meeting goals is not imposed; as guidelines, 
certain criteria of MAP must be monitored constantly; meeting goals one 
year would not presume they are met the following year. The second goal 
of the self-study is that program staff members view MAP as a work-in- 
progress, that is, they are expected to add new criteria over the course of 
the years to reflect a program’s changing priorities. In keeping with these 
goals, a complete self-study will be required of program sites in advance 
of a cyclical review, a phase similar to the self-study that initiates an 
accreditation review at a college or university department or program. 
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Some Centers used the self-study as the basis for funding priorities 
for the coming year, 1999-2000, particularly to enhance the Student 
Learning Environment. Curriculum development includes such initiatives 
as Music History and Theory courses in Vienna, an extended Orientation 
with intensive language study in Salamanca to facilitate enrollment in 
Universidad de Salamanca, and revised language materials in Beijing, 
with training in their use for instructors. Activities other than coursework 
that support cultural integration, including interaction with students in 
the host country, field study, and membership in academic or profession- 
al associations for the benefit of students also count among funding prior- 
ities. IES Madrid will initiate a Cinema Forum that will enlarge the pro- 
gram of intercambios to include viewings and discussions of classic Spanish 
films with a professor of film at Universidad Complutense de Madrid. IES 
Paris is organizing a required field trip to Marseilles and an optional field 
trip to either Senegal or Benin, to complement the new Francophone 
Studies curriculum. In London, an invitation to join the Royal Institute 
of International Affairs is being funded, which will give students the 
opportunity to attend public lectures and use the private library. 

Conclusion 

The goals of this project were twofold: to develop a product — a set 
of criteria of quality assurance — and to put in place a process for continu- 
al systematic evaluation, both self-evaluation and the periodic program 
reviews by outside evaluators. As the project evolved, it also drew on the 
principles of organizational learning, specifically, involving employees in 
planning and implementing change. In theory, involving an organiza- 
tion’s members in the evaluation contributes to the individual’s and the 
organization’s learning (Torres, Preskill and Piontek, 3). The development 
of MAP challenged staff members to think specifically about the meaning 
of academic quality in study abroad programs and methods for maintain- 
ing and improving academic quality. Conversely, the criteria also offer a 
framework for defining academic quality in terms of the goals of the orga- 
nization and the specific mission of each Center. 

Organizational learning will continue to develop through the self- 
study conducted by Center staff members in conversation with IES 
Chicago staff members and through student evaluations. “As new learn- 
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ing emerges from collecting, analyzing and interpreting data, the evalua- 
tive inquiry process is recycled by asking additional questions, identify- 
ing values, beliefs and assumptions and reflecting on, and engaging in 
dialogue, about the findings. The continuous process of evaluative inquiry 
further nourishes individual, team and organizational learning” (Preskill 
and Torres, 3). In this particular setting, the process carries implications 
for the organization’s strategic planning for academic quality. The criteria 
of MAP serve as talking points for allocating financial resources; that is, 
the evaluation of academic quality is directly tied to decisions to provide 
funds to enhance academic quality. This causal connection between eval- 
uation and the allocation of resources likewise establishes a link between 
resources and the quality of the student learning environment. In a cli- 
mate of organizational learning, the organization’s members — staff, facul- 
ty, and students — who engage in inclusive discussions about academic 
quality are the ones who benefit. 

Notes 

1 The authors wish to thank the members of the IES Board of 
Trustees for providing support for this project through the endowment 
fund and Mary Dwyer, President, IES, for her leadership and vision. The 
authors also acknowledge with gratitude the assistance of the members of 
the Task Force: IES Board members Stanley Katz, Woodrow Wilson 
School, Princeton University and Kathryn M. Moore, College of 
Education, Mighigan State University; affiliate representatives David 
Deal, Department of History, Whitman College; Pamela Ferguson, 
Department of Mathematics, Grinnell College; Mark Ruhl, Department 
of Political Science, Dickinson College; and Michael Steinberg, Vice 
President for Academic Programs, IES. 

9 

- Larry Braskamp, Muriel Poston, and Jon Wergin identify four sys- 
tems that work together to insure public accountability and the improve- 
ment of academic quality: internal campus reviews; governmental laws, 
rules and regulations; the marketplace; and accreditation. Of these four, 
internal reviews and accreditation focus on individual institutions. “By 
respecting unique institutional and program missions accreditation has 
preserved what has become the envy of the world: a rich diverse array of 
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colleges and universities Thus accreditation has nurtured intellectual 

capital, the pursuit of scholarship relatively unfettered by external poli- 
tics, extreme pragmatism, and power” in “Accreditation: Sitting Beside or 
Standing Over?” ( CHEA , 1997) 1-2. Serbrenia J. Sims identifies the ori- 
gin of this principle as the North Central Association, which determined 
in the 1930s to judge an institution “in terms of its purpose and total pat- 
tern as an institution,” thus making it possible to adapt the accrediting 
process to a variety of schools in Student Outcomes Assessment: A Historical 
Review and Guide to Program Development (New York: Greenwood Press, 
1992) 69-70. 

3 Ronald Pirog and Patricia Martin focus on programs sponsored by 
U.S. institutions of higher learning, but acknowledge that “there are so 
many different types of (overseas education) programs in existence, rang- 
ing from U.S. -planned and run traditional ‘island’ programs and branch 
campuses to direct enrollment in foreign universities and exchanges — as 
well as experiential, service-learning, and internship programs. This 
superabundance of numbers and types makes the task (of evaluation) 
extremely difficult — especially if one wants to compare the quality of one 
to another” in NAFSA's Guide to Education Abroad for Advisers and 
Administrators 2nd Ed. (Washington, D.C.: NAFSA, 1997) 333. 

^ David R. Schwandt defines organizational learning as an institu- 
tional way of life, one that makes long-term contributions to the organi- 
zation’s health and adaptability. Based on the idea of the organization as a 
social system, the concept applies to the particular system “of actions, 
actors, symbols, and processes that enable an organization to transform 
information into valued knowledge which in turn, increases its long-run 
adaptive capacity” in “Learning as an Organization: A Journey into 
Chaos.” Ed. Sarita Chawla and John Renasch. Learning Organizations: 
Developing Cultures for Tomorrow's Workplace (Portland: Productivity Press, 
1995) 370. Rosalie T. Torres, Hallie S. Preskill and Mary E. Piontek 
explain the theory in practice as “a continuous process of organizational 
growth and improvement that (1) is integrated with work activities; (b) 
involves the alignment of values, attitudes, and perceptions among orga- 
nizational members; and (c) uses information or feedback about both 
processes and outcome to make changes” in Evaluation Strategies for 
Communicating and Reporting (Thousand Oaks, Cal.: Sage, 1996) 2. 

^ Larry A. Braskamp and David C. Braskamp explain the current 
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philosophy that “colleges and universities are now being conceptualized as 
learning communities — communities in which the most important per- 
sons are the students as learners. Faculty and the surrounding learning 
environment — the curriculum, social activities, facilities — exist to foster 
the social, physical, spiritual, and the intellectual development of stu- 
dents. . . When standards are connected to student performance they pro- 
vide a very compelling argument for refocusing the definition of quality 
in higher education. By linking standards and performance, student learn- 
ing and development becomes the starting point for examining program 
quality (i.e., program and institutional effectiveness)” in “The Pendulum 
Swing of Standards and Evidence” ( CHEA Chronicle 5, 1997) 1. Alexander 
Astin similarly advances the concept of academic quality in terms of 
“enhancing the intellectual and scholarly development” of both faculty 
and students. “The most excellent institutions are, in this view, those that 
have the greatest impact — ‘add the most value,’ as economists would 
say — to the students’ knowledge and personal development” in Assessment 
for Excellence: The Philosophy and Practice of Assessment and Evaluation in 
Higher Education (NY: American Council on Education, Macmillan 
Publishing Co., 1991) 6. 

^ Patrick T. Ewell references a joint project of the National Center 
for Higher Education Management Systems and Education Commission 
of the States that queried corporate and political leaders in focus groups 
on the issue of “what the results of four years of college ought to be.” 
Among the characteristics identified by the groups was “the ability to 
bridge cultural and linguistic barriers. While in some ways resembling 
the academy’s own concern with cultural diversity, the emphasis here was 
again far more applied. At its center was the ability to work effectively 
with people drawn from different backgrounds and value systems, far 
more than simply personal tolerance for ethnic and national differences” 
in “Assessment of Higher Education Quality: Promise and Politics.” ed. 
Samuel J. Messick, Assessment in Higher Education: Issues of Access, Quality, 
Student Development and Public Policy (Mahwah, N.J.: Lawrence Erlbaum 
Associates, Publishers, 1999) 151-152. 

Sims describes the difference between summative and formative 
data. Formative data, that gathered by the Center self-assessments and 
student evaluations of courses and programs, provides “information about 
how a program operates and how to improve it” and the main audiences 
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are program staff and managers. The main use of summative data, the 
Center profiles, is administrative: that is, to aid in decisions such as the 
allocation of funds, for audiences that include the organization’s leaders 
and sponsors (86). 


APPENDIX 

IES MODEL ASSESSMENT PRACTICE 

I. Student Learning Environment 

A. Pre-departure information 

1. Information presented to students includes practical informa- 
tion about academic program requirements, living and trav- 
el arrangements, and expectations about expenses 

2. Information begins to prepare students for the challenge of 
crossing cultures 

B. Cultural orientation 

1 . The Center conducts an extensive initial orientation program 
for its students that includes information on gender and race 
relations in the host country and on developing skills in 
intercultural competence 

2. Staff and guest speakers knowledgeable about the local cul- 
ture conduct the orientation 

3. Under appropriate supervision, students are given immediate 
opportunities to explore and function in the local setting 
(e.g., travel, eating, attendance at local cultural events) 

4. Center provides on-going opportunities to discuss cultural 
adaptation 

C. Instructional quality 

1 . Faculty are evaluated by student surveys for each course they 
teach and Center directors review evaluations 

2. Faculty expectations of student work are rigorous 

3. Faculty participate in orientation programs on learning styles 
and expectations of American students and appropriate 
teaching strategies 

4. Faculty utilize instructional technology when appropriate in 
their teaching 

5. Faculty are encouraged to meet formally and informally to 
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discuss pedagogy, course content, and student progress 

6. Class experiences make effective use of location 

7. Language instruction, when appropriate, makes effective use 
of field study and local cultural institutions 

D. Curricular design and integration 

1 . Academic programs are designed to enhance student engage- 
ment in the intellectual, political, cultural, and social insti- 
tutions of the host country 

2. Course content and curriculum adequately reflects the variety 
of cultures of the host country 

3. Out of classroom activities are integrated with in-class course 
work 

4. Sequence of courses and learning experiences are consistent 
with the student’s academic programs in America 

5 . The curriculum meets the interests of current and potential 
IES students 

E. Curricular focus on intercultural competence 

1 . Approaches to course material should incorporate an analysis 
of cultural differences 

2. Students are strongly encouraged in appropriate courses to 
compare the institutions of the host country with similar 
institutions in America 

F. Language development opportunities 

1 . Language instruction, when appropriate, is integrated into all 
courses and IES activities 

2. In language development courses, students gain a perspective 
of the host country’s culture, values, history, and current sta- 
tus 

3. Students are provided out-of-classroom opportunities to 
develop oral, listening, and writing skills in the language of 
the host country 

4. The center environment strongly encourages use of host 
country language 

G. Internships 

1 . Internships give IES students the opportunity to participate 
in and to critically observe a segment of the work force in the 
host country 
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2. Internships make effective use of location, local talent, and 
local resources 

3. Internships help develop intercultural cognitive and inter- 
personal skills 

4. Internships are conceived as community-based learning and 
require students to synthesize the practical and theoretical 
aspects of their work site 

5. Student time and energy spent in internships are assigned 
appropriate academic credit 

H. Field study and trips 

1. Field studies reinforce Center’s academic goals and/or devel- 
opment of intercultural competency skills 

2. Supervision of field study is integrated into the academic pro- 
gram 

3. Guided field trips help students take advantage of the region 
and result in more learning than if students attempt to trav- 
el on their own 

I. Engagement in IES-sponsored cultural and social activities 

1 . The Center organizes activities that facilitate student engage- 
ment in the local culture 

J. Involvement in other academic, political, economic and cultural 
institutions 

1 . Students are given guidance and directions for involvement 

2. Representatives of local institutions are members of the faculty 

3. Requirements for minimal student participation are a part of 
the course work 

4. The Center provides assistance to students who enroll in 
courses at the local university 

K. Reentry into home culture and home institution 

1. The Center offers reentry programs to students about possi- 
ble difficulties adjusting to their home campus 

2. The Center encourages students to be ambassadors for the 
country where they studied 

3. The Center provides an environment for reflecting on and shar- 
ing the cognitive and intrapersonal aspects of their experience 

II. Student Learning: Assessment and the Development of Intercultural Competency 

A. Intellectual Development 
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1. Students demonstrate that they have acquired substantial 
knowledge and understanding of course material in IES 
courses and courses at local universities 

2. Students develop their skills in critical thinking through pro- 
grammed exposure to political, cultural and social institu- 
tions of the host country 

3. Students develop their ability to understand and critique 
their own value system through contact with a variety of cul- 
tural perspectives in the host country 

B. Development of language and communication skills 

1. Students systematically report on how and what they have 
learned about the host country’s language 

2. Students report their interactions with members of the host 
culture to IES staff 

3. Students engage in periodic self-evaluation of their commu- 
nication skills in the language of the host culture during 
their course of study 

4. IES sponsors periodic oral proficiency interviews to measure 
oral skills according to American Council of Teachers of 
Foreign Languages or similar guidelines 

5. IES language faculty prepare students at highest language 
levels for appropriate in-country exams that demonstrate 
mastery 

C. Cognitive growth 

1. Students are assessed for their knowledge of cross-cultural 
comparisons 

2. Assessment of student knowledge and development are used 
to enhance the curriculum 

3. Students develop an increased ability to recognize cultural 
difference 

4. Students acquire a greater appreciation and respect for per- 
sons with differing cultural values 

D. Interpersonal growth 

1 . Reports of the students’ ability to live comfortably in a dif- 
ferent culture are gathered routinely as part of semester-end, 
general program evaluation 

2. Students are able to adapt to the culture of the host country 
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3. Students became more comfortable interacting with persons 
of different backgrounds 

4. Students acquire a greater appreciation and respect for per- 
sons with differing culturally-based values 

5. Students acquire general adaptive skills that prepare them to 
live in a variety of foreign cultures 

E. Intrapersonal growth 

1 . Students and IES Center staff report that students are able to 
make their own decisions and reflect on them 

2. Students and IES Center staff report that students have 
gained a better understanding and acceptance of their unique 
values and capacities 

3. Students self-report on insights and self-analyses of their per- 
sonal development 

4. Students are assessed on their level of comfort with explo- 
ration and experimentation and what they learned from the 
new experiences 

5. Students demonstrate that they are more able to take care of 
their concerns as they progress through the program 

6. Students report on their ability to develop attitudes and skills 
that facilitate life long learning 

III. Resources for Academic and Student Support 
A. Faculty qualifications 

1. Faculty have academic credibility and credentials 

2. Faculty who are language instructors are qualified to teach at 
a local university or university preparatory institutions 

3. The scholarly achievements of the IES academic faculty meet 
local university or equivalent standards 

4. Academic faculty generally are currently engaged in scholar- 
ship 

5. Faculty are selected to teach IES courses based on their abili- 
ty to teach and their commitment to the IES goals 

6. Faculty are involved in developing new courses according to 
the Center’s curriculum design for approval by the 
Curriculum Committee 

7 . Faculty are sensitive to gender and cultural differences among 
students 
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8. Professionals who teach IES classes have relevant experiences 
in their field of expertise 

B. Administrative staff qualifications 

1 . Center director and staff are collaborative and mutually rein- 
forcing in meeting student needs 

2. Center director and staff are courteous, sensitive, and accom- 
modating to student needs 

3. Center director and staff are committed to study abroad and 
the development of intercultural competence skills 

4. Center director and staff are interested in and able to work 
with American undergraduates 

5 . Center director and staff members participate in annual per- 
formance evaluations 

6. Center director has appropriate administrative experience 

7. Center director has credibility at local universities 

8. Center director is proficient in the English language and the 
language of the host country 

9. Center director has an appropriate knowledge of the academ- 
ic expectations of American colleges and universities and 
plans and administers the center program in that context 

10. Center director adequately responds to the annual evaluation 

11. Persons under contract who are responsible for accounting, 
internship supervision, and/or housing arrangements are suf- 
ficiently qualified 

C. Academic advising and counseling 

1 . Advisors to students on course requirements have the neces- 
sary academic qualifications 

2. Advisors to students are knowledgeable about IES courses 
and requirements 

3. Advisors to students on university enrollment are knowl- 
edgeable about local university rules and requirements 

4. Advisors are familiar with the IES Academic Guidelines 

5 . Advisors are readily available to students 

6. Center staff are sufficiently knowledgeable about the local 
academic requirements to appropriately assist students 

7. Center staff are adequately trained to deal with personal con- 
cerns of the students 
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8. Center staff know of available IES resources for referral 

D. Center staff size 

1 . The ratio of staff- to-students is in the range of 1 to 1 5 

2. The size of the staff is appropriate for the types of programs 
offered (e.g., internships) 

E. Center facilities 

1 . Center director has a private office for consultation 

2. Faculty have a desk in a private room for advising students 

3. Students have a place where they can gather and meet infor- 
mally 

4. Classrooms are adequate for the IES courses 

5. The classrooms are equipped with adequate instructional 
technology 

6. The Center is well located for student access to the local cul- 
ture 

F. Access to local educational and cultural institutions 

1. IES Center has written agreements with universities for 
course enrollment and credit 

2. IES Center has agreements (preferably written) with univer- 
sities for access to sponsored activities and student clubs 

3. IES Center has agreements (preferably written) with univer- 
sities or other agencies for access to sponsored sports activi- 
ties 

4. IES provides information regarding cultural opportunities 

G. Library resources 

1 . Library contains up-to-date reference books 

2. Students have access to one or more libraries at local univer- 
sities 

3. Students are adequately informed and encouraged by IES to 
take advantage of research resources available to them 

4. Students have access to specific collections necessary for class 
assignments 

5. Library hours are convenient for students, within bounds of 
building security 

6. Books and periodicals are adequate for students to complete 
the course requirements 

H. Instructional technology 
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1 . Students have access to computers in a quiet working area 

2. Students have access to e-mail 

3. Students have access to collection of videos, slides, music, and 
language tapes or CDs 

4. Students have adequate access to the Internet, databases, and 
other digital resources 

5. Instructional technology is available for faculty use 

6. Faculty have use of photocopy machine for coursework 

I. Housing and home stays 

1 . Students have a safe place to live 

2. Public transportation is readily available 

3. Housing arrangements with families and/or students of host 
country are based on the opportunity for high level of inter- 
action 

4. Contracts with housing providers adhere to laws of host coun- 
try 

5 . Housing is well located for student access to local culture 

6. Housing provides students with study space 

J. Student qualifications 

1. Students have sufficient knowledge of the language of the 
host country before enrolling 

2. Students meet minimal GPA as specified by IES 

3. Students have sufficient academic preparation at the home 
institution before enrolling at an IES Center 

4. Admissions process does not discriminate against students 
with disabilities 

K. Health and safety 

1 . Students have adequate health insurance 

2. Students have access to good health care 

3. Students are notified about what to do in an emergency 

4. Center staff members are informed about local health and 
safety concerns 

5. Center staff members follow IES guidelines on health and 
safety 

6. Center staff members enforce IES policy on sexual harassment 
IV. Interactions among IES Chicago Office, IES centers, and colleges and univer- 
sities in the United States. 
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A. Quality of communications 

1 . IES Chicago Office and Center staff share timely information 
on resource demands and allocation 

2. IES Center Staff appropriately consult with the Chicago office 
about staff and faculty decisions 

3. Both IES Chicago and Center staff participate in short and 
long-term planning 

B. Involvement of U.S. college faculty 

1 . IES Curriculum Committee is routinely consulted on matters 
of curriculum development 

2. Faculty are active participants in the periodic academic pro- 
gram reviews 

C. Efficiency of recruitment and enrollment procedures 

1. IES Chicago Office accurately informs the students of IES 
requirements, costs, and opportunities 

2. IES Chicago Office facilitates interaction and sharing of infor- 
mation among the IES Centers 

D. Professional development 

1 . Center director builds and maintains relationships with local 
academic community 

2. IES faculty are involved in developing new courses according 
to the Center’s curriculum design for approval by the 
Curriculum Committee 

3. IES Chicago Office promotes relationships between IES 
Centers and U.S. college and university faculty (eg. Teaching 
assistants, faculty seminars, faculty associates) 

4. IES encourages and supports U.S. colleges in developing pro- 
grams at their institutions when possible on orientation to 
practical living matters abroad and local cultural expecta- 
tions and on re-entry 
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